Repository navigation
Metrics files next to the model, and release 1.13.0 - #46
Merged
Merged
Conversation
A metrics file, `<stem>.metrics.yml` beside the model, says what the model's numbers mean: simple metrics (a column aggregated), row counts, ratios of totals and derived metrics (arithmetic over metrics), with filters that reach other tables through many-to-one foreign keys along exactly one path, and nulls as in SQL. Every `measure` column of the model also implies a metric, named as the studio's semantic export names it. - model2data/spec/metrics/: the normative README and JSON Schema, versioned on its own (0.1.0), and a coffee_webshop example that conforms against the example model (which now marks total_amount and quantity as measures). - model2data.metrics: the reader (the model spec's YAML profile, its Issue type and schema messages), the checks within the file and against the model, one resolved semantic representation (entities, relationships, dimensions, measures, metrics with their join paths and time), known values computed on DuckDB with the SQL the dbt tests run, a singular dbt test per metric, and an Apache Ossie 0.1.1 export with a lossiness report. The generator never reads a metrics file. Co-Authored-By: Claude Opus 5.5 <[email protected]>
…st|export - `validate` checks a *.metrics.yml file against the metrics spec and its model: `--model`, else the model beside it with the same stem. - `generate --metrics FILE` adds metric_values.json (each metric's known value over the seeds, overall and by month), a singular dbt test per metric under data-tests/metrics/ and osi/<project>.yml (Apache Ossie 0.1.1), and changes no other file. A sibling metrics file is not picked up on its own, so an unchanged command keeps writing the same files. - `metrics list` and `metrics export --to ossie` work without generating; with no metrics file they use the model's inferred metrics. The determinism suite gains two cases with metrics; every existing hash is unchanged. The dbt cases run `dbt build` on the reference example, a model with every kind of column and join, and the training model with defects over several days, and require every metric test to pass. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A "Metrics next to the model" section in both READMEs, with the coffee example, what the known values and the metric tests are, and the Ossie export; the metrics file and commands in LLMS.md and `guide tune`, and what to do about their messages in `guide triage`. The guide's option check now reads the options of grouped commands (`metrics export --to`). Co-Authored-By: Claude Opus 5.5 <[email protected]>
Metrics next to the model: metrics spec 0.1.0, `validate` on metrics files, `generate --metrics` (known values, a dbt test per metric, Apache Ossie 0.1.1), `metrics list|export`, and the `model2data.metrics` API. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
The README now leads with what model2data does today: one line per capability with a link to its page, a before and after of a real run with metrics (model and metrics in; the project, dbt build's summary and metric_values.json out), who it is for, and how it works with the metrics step in the diagram. README_PYPI.md carries the same top half with absolute links and a text diagram. The reference moved to docs/, one page per topic, worded as it was. Two fixes on the way: the coffee_webshop metrics command now passes --rows 200 (the values quoted are for 200 rows), and the rounding line says what metric_values.json says (integer sums, minimums and maximums are exact). examples/ecommerce.metrics.yml gives the e-commerce example revenue, orders, average order value and units sold, and the demo GIF shows them: generate with --metrics, dbt build with the four metric tests passing, the known values. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an optional metrics file next to the model,
<stem>.metrics.yml, with its own spec (metrics spec 0.1.0) and JSON Schema. It is the engine half of "metrics next to the model", decided on 6 Oct 2026; the studio builds its Metrics tab on top of it next. The full list is in the 1.13.0 CHANGELOG entry.What it adds
The format. Four kinds of metric:
expression), the one documented exception to "nothing encoded in a string".Simple metrics and row counts take a
wherefilter witheq,ne,in,not_in,gt,gte,lt,lte,betweenandis_null, combined withanyandall, with nulls as in SQL. A filter may name a column on any table the metric's table reaches many-to-one, along exactly one path. Everymeasurecolumn also yields an inferred metric, named as the studio's semantic export names them today.model2data.metrics, the public API:load/validate, andresolveinto one semantic representation (entities, relationships, dimensions, measures, metrics with their join paths). Alsoknown_values,to_ossieandwrite_metric_tests.Known values. Computed with DuckDB over the generated data, overall and per month, with the same SQL the dbt tests run. Counts are exact; everything else is rounded to 6 places, half to even.
CLI.
validatechecks*.metrics.yml, against the model beside it by default.generate --metrics FILEaddsmetric_values.json, one dbt test per metric indata-tests/metrics/, and an Ossie file inosi/.metrics listandmetrics export --to ossiework without generating.Apache Ossie export, 0.1.1 as tagged
osi-0.1.1-rc1. No final 0.1.1 tag exists yet. What Ossie has no field for goes intocustom_extensionsand is listed at the top of the file. Every example is checked against the vendored Ossie schema (Apache-2.0, with its LICENSE and NOTICE).Docs: README, README_PYPI, LLMS.md, the
guidetext, the spec README, PUBLISHING.md rows for/spec/metrics/0.1.0/, and the CHANGELOG.Determinism
determinism.jsonmoved; two cases were added.--metricsonly adds files.test_metrics_only_add_fileschecks that every file shared with a run without it is byte-identical.Decisions made while building
format: percentexpects a fraction (0.6 shows as 60%), as BI tools do, and is never multiplied.data-tests/metrics/, because that is the generated project'stest-paths.generatedoes not pick up a metrics file beside the model on its own, so an unchanged command keeps writing the same files.staging.stg_<table>).Checked
dbt buildon four cases passes every metric test, including defects over several days. A known value moved past its tolerance makes its test fail.Not verified
osi/folder, and Snowflake's OSI import.After merge: publish the v1.13.0 release (PyPI), and host
/model2data/spec/metrics/0.1.0/on the website per PUBLISHING.md.🤖 Generated with Claude Code